The structure of information: from probability to homology

D. Bennequin and P. Baudot introduced a cohomological construction adapted to information theory, called ”information cohomology” (see ”The homological nature of Entropy”, 2015). Our text serves as a detailed introduction to information cohomology, containing the necessary background in probability theory and homological algebra. It makes explicit the link with topos theory, as introduced by Grothendieck, Verdier and their collaborators in the SGA IV. It also contains several new constructions and results. (1) We define generalized information structures, as categories of finite random variables related by a notion of extension or refinement; probability spaces are models (or representations) for these general structures. Generalized information structures form a category with finite products and coproducts. We prove that information cohomology is invariant under isomorphisms of generalized structures. (2) We prove that the relatively-free bar construction gives a projective object for the computation of cohomology. (3) We provide detailed computations of H1 and describe the ”degenerate” cases. (4) We establish the homological nature of Tsallis entropy. (5) We re-interpret Shannon’s axioms for a ’measure of choice’ in the light of this theory and provide a combinatorial justification for his recurrence formula.